Skip to content

ENH: Add support for type annotations #601

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 14 commits into
base: main
Choose a base branch
from

Conversation

RastislavTuranyi
Copy link

@RastislavTuranyi RastislavTuranyi commented Jan 15, 2025

Resolves #196

Hi! I ran into the issue that numpydoc does not support typehints before Christmas, so I decided to have a crack at it. I have tried multiple different approaches (not in this git history), especially trying to combine numpydoc with sphinx-autodoc-typehints, but what I finally settled on is to make use of numpydoc's powerful class system (I have found that the data representation of numpydoc and sphinx-autodoc-typehints are largely incompatible).

My implementation edits the _parse_param_list method - responsible for creating the docstrings for Parameters, Other Parameters, Attributes, Methods, Returns, Yields, Raises, and Receives sections - so that a type can be obtained from the type hint if one is not specified in the docstring (btw, I believe it should be relatively simple to implement #356 following a similar method). This means that the type hints can be supported for all those sections with minimal code. The way this works is defined in a new method, _get_type_from_signature, which by default on NumpyDocString only returns an empty string (no type). The actual type inference is implemented on the subclasses:

FunctionDoc

Getting type hints for functions is the easier task since we can rely fully on inspect.signature. Firstly, the signature is obtained in __init__ to avoid recomputing it at every iteration of _parse_param_list. All that is then necessary in _get_type_from_signature is to return the type hint associated with the argument name, provided that the function has a signature and that the type hint is set.

There is a small complication in that multiple parameters can be combined into one entry, but all that is necessary to support that is to ensure that all the combined parameters are type-hinted as the same type, otherwise return no type.

ClassDoc

At the basic level, the type hints for classes are similar to FunctionDoc and the code is therefore repeated (there is potentially room for optimisation where the code could be pulled out to NumpyDocString, should that be desirable). However, there is an additional complication in that the class documentation may contain the attributes (which may be @property) and therefore are not part of the __init__ signature. This required an additional method, _find_type_hints:

  • First is handled the case where the arg_name is an attribute with a type annotation (typing.get_type_hints handles unwrapping etc.)
    • Note that the _annotation_to_string function handles the various possiblities for what the annotation can be - missing, a class instance, or a type from the typing module.
        type_hints = get_type_hints(obj, include_extras=True)
        try:
            annotation = type_hints[arg_name]
        except KeyError:
            ...

        return _annotation_to_string(annotation)

def _annotation_to_string(annotation) -> str:
    if annotation == inspect.Signature.empty:
        return ""
    elif type(annotation) is type:
        return str(annotation.__name__)
    else:
        return str(annotation)
  • In the ..., the remaining options are that the arg_name doesn't exist, handled via:
            try:
                attr = getattr(obj, arg_name)
            except AttributeError:
                return ""
  • Or that it is a property, in which case inspect.signature is used to obtain the return type (note that in the case of non-standard properties, this block is not triggered):
            if isinstant(attr, property):
                try:
                    signature = inspect.signature(attr.fget)
                except ValueError:
                    return ""

                return _annotation_to_string(signature.return_annotation)
  • Or that it is a class attribute without an annotation (but with the value set), in which case it can be extracted:
            return type(attr).__name__

Once again, though, the above is further complicated by the fact parameters can be combined (not sure whether it is valid to combine attributes as well, but this implementation can handle that as well). This is solved again by ensuring all the combined parameters/attributes are of the same type, as returned by _find_type_hints.

Robustness

From my testing, this implementation is able to handle everything my other projects could throw at it, but I would not be surprised if there are edge cases where it breaks down. Regardless, though, I think it might be possible to increase the robustness of my implementation by using some of the type-extraction functions from sphinx-autodoc-typehints (whether by depending on it as a whole, or by copying the relevant functions, I am unsure), such as get_annotation_args or format_annotation.

Conclusion

Please let me know what you think! I am keen to get numpydoc working with type annotations, and I believe that this implementation is a straightforward way to do that, but I am curious what your plans are for this issue.

Notes

  • Support type annotations in Returns and Yields sections #356 return type hints is not implemented here since I think it requires more discussion about the spec, but happy to do that as well
  • this implementation only inserts type, not the default values, but adding the defaults from the signature as well should be simple - not sure whether to add it here or separate PR though

RastislavTuranyi and others added 4 commits January 4, 2025 12:28
For some functions, the signature cannot be obtained and so inspect.signature raises a ValueError, which can crash the sphinx build. This has been fixed by catching the exception
For ClassDoc, the _get_type_from_signature method handles not only the docs for __init__, but also for the Attribute and Method etc. sections, whose typehints cannot be obtained from signature(__init__). Therefore, further code has been added that attempts to find the type hint from different sources and should work for functions, methods, properties, and attributes.
ForwardRef is no longer necessary since the functionality is handled via typing.get_type_hints
Rastislav Turanyi added 9 commits May 16, 2025 10:19
Also fixes the function type hints for cases where annotations were not imported from __future__
Also fixes class parameters type hints without annotations and enables full Annotated rendering for attributes
Also introduces a refactor, pulling out a helper function that turns an annotation to a string
Tests both when parameters with the same type are combined and when parameters of different types are combined. Includes a refactor where the _handle_combined_parameters method was pulled into the base class
The _find_type_hint method now provides better type inference for class attributes without type hints at the cost of less useful types for the Methods section. However, that section does not seem to be using the types in the final Sphinx render and it is not clear that types even make sense there, so it might be a reasonable trade-off.
@RastislavTuranyi
Copy link
Author

Assuming no news is good news, I have gone ahead and wrote the missing tests (as well as fixed the failing ones)! There were also some changes in response to test failures (original PR updated), the biggest of which concerns method types and class attribute type inference:

  • Previously, the type hint inserted for:
    • methods in the methods section would be the return type of the method, and
    • class attributes without type hint would be:
      • the return type if the value is a callable (function, method, class, etc.)
      • the type of the value otherwise
  • Now, the type hint inserted for:
    • methods in the methods section is function, and
    • class attributes without type hint is the type of the value

Now, this should be an improvement since as far as I can see, the type hints for methods are not used at all, so it doesn't seem to matter what they are and the type inference is just better, but it raises two important questions:

  • Should we be adding a type to the methods at all?
  • Should we be doing the type inference if class attributes have not been type-hinted?

@RastislavTuranyi RastislavTuranyi marked this pull request as ready for review May 19, 2025 08:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support type annotations (PEP 484)
1 participant